Assessment system and method

ABSTRACT

An assessment system and method configured to automatically assess an individual and an associated job using a computing device. The system includes a benchmark module that stores a benchmark associated with a job. The benchmark includes both minimum threshold scores and maximum threshold scores associated with particular characteristics. The system includes an electronic library of questions associated with an array of characteristics. The system includes a survey module functionally coupled to the electronic library and the benchmark module that automatically performs an assessment by automatically selecting questions from the electronic library based on the benchmark and applying such questions to an individual using a user interface module.

CROSS-REFERENCE TO RELATED APPLICATIONS

This invention claims priority, under 35 U.S.C. § 120, to the U.S.Nonprovisional patent application Ser. No. 14,1993,432 to Benjamin F.Shaw et al. filed on Jan. 12, 2016, and therethrough, to the U.S.Provisional Patent Application No. 62/102,334 to Benjamin F. Shaw et al.filed on Jan. 12, 2015, both of which are incorporated by referenceherein in their entirety.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to assessment systems and methods,specifically an assessment system and method to assessing candidates andthe like with positions and the like.

Description of the Related Art

Finding the ‘right job and/or, as an employer, finding the ‘right’employee are critical pursuits of most people in developed nations.There are various types of support for these pursuits.

An employment agency is an organization which matches employers toemployees. In substantially all developed countries, there is a publiclyfunded employment agency and a multiplicity of private businesses whichact as employment agencies and/or perform similar functions/services. Anexecutive-search firm specializes in recruiting executive personnel forcompanies in various industries. This term may apply tojob-search-consulting firms who charge job candidates a fee and hospecialize in mid-to-upper-level executives. In the United States, somestates require job-search-consulting firms to be licensed as employmentagencies.

Some third-party recruiters work on their own, while others operatethrough an agency, acting as direct contacts between client companiesand the job candidates they recruit. They can specialize in clientrelationships only (sales or business development), in findingcandidates (recruiting or sourcing), or in both areas. Most recruiterstend to specialize in either permanent, full-time, direct-hire positionsor in contract positions, but occasionally in more than one. In anexecutive-search assignment, the employee-gaining client company—not theperson being hired—pays the search firm its fee.

Recruitment/hiring can be said to be the overall process of attracting,selecting and appointing suitable candidates for jobs (either permanentor temporary) within an organization. Recruitment can also refer toprocesses involved in choosing individuals for unpaid positions, such asvoluntary roles or unpaid trainee roles. Managers, human resourcegeneralists and recruitment specialists may be tasked with carrying outrecruitment, but in some cases public-sector employment agencies,commercial recruitment agencies, or specialist search consultancies areused to undertake parts of the process. Internet-based technologies tosupport all aspects of recruitment have become widespread.

In situations where multiple new jobs are created and recruited for thefirst time, or the nature of a job has substantially changed, a jobanalysis might be undertaken to document the knowledge, skills,abilities and other characteristics required or sought for the job. Fromthese relevant information is captured in such documents as jobdescriptions and job specifications. Often, a company already has jobdescriptions for existing positions. Where already drawn up, thesedocuments ma require review and updating to reflect currentrequirements. Prior to the recruitment stage, a person specificationshould be finalized to provide the recruiters with the project'srequirements and objectives.

Various psychological assessments and tests can assess a variety ofcharacteristics, including literacy, critical thinking, attention todetail, personality traits, tendencies, preferences, and the like. Testsare also available to measure physical ability. Recruiters and agenciesmay use applicant tracking systems to filter candidates, along withsoftware tools for psychometric testing and performance-basedassessment. In many countries, employers are legally mandated to ensuretheir screening and selection processes meet equal opportunity andethical standards.

Employers are likely to recognize the value of candidates who encompasssoft skills such as interpersonal or team leadership. Many companies,including multinational organizations and those that recruit from arange of nationalities, are also often concerned about whethercandidates fits the prevailing company culture.

Some improvements have been made in the field. Examples of referencesrelated to the present invention are described below in their own words,and the supporting teachings of each reference are incorporated byreference herein:

U.S. Pat. No.: 8,543,414, issued to Iwano et al., discloses a medicalquestion contents automatic selection system for automatically selectingmedical questions allocated to a patient from the medical questionnairemade by a medical professional. The medical question contents automaticselection system compares the correct answer rate threshold of themedical questions preset by the medical professional with the correctanswer rate of the questions to the patient so as to estimate theeducational level of the patient about health, compares the vital signmeasurement value threshold preset by the medical professional with thevital sign measurement value of the patient so as to estimate thecondition level of the patient, and automatically selects medicalquestions allocated to the patient depending on the educational leveland the condition level of the patient. The medical question contentsautomatic selection system can allocate medical questions to the patientdepending on the educational level of the patient about health anddisorder condition level of the patient without troubling the medicalprofessional.

U.S. Pat. No. 8,156,051, issued to Shah et al., discloses recruiting orthe process of locating and qualifying potential candidates foremployment is largely a manual process consuming significant resources.The employment recruiting system automates many of the services relatedto employment recruiting. Position advertisements and candidate resumesare stored in computer searchable data bases. A data processing deviceautomatically searches the data base containing candidate resumes inresponse to a search parameter associated with a position advertisement.The system automatically contacts screens candidates.

U.S. Pat. No. 8,112,365, issued to Foster, discloses a system and methodfor online employment recruiting and evaluation is provided. An online,web-based environment allows employers to create job descriptions forposting on an Internet-based job search service in communication withthe web-based environment, and allows for online recruitment and reviewof hiring candidates using a multi-phase approach. Eachemployer/supervisor specified by the candidate is interviewed, and theinterview is recorded and is stored as part of the candidate's virtualdossier.

U.S. Patent Application Publication No.: 2004/0138903, by Zuniga,discloses an employment management tool has an employment managementprogram. The employment management program has a notification component,an evaluation component, a form server and a process manager. A serveris coupled to a data network and runs the employment management program.A database is coupled to the server.

U.S. Patent Application Publication No.: 2006/0080128, by Brow et al.,discloses a method, system, and computer program receive anidentification of one or more skills associated with a job. The method,system, and computer program also receive a selection of one or morequestions associated with the one or more skills. In addition, themethod, system, and computer program provide an interview form, whichincludes the selected questions for use during an employment interview.

The inventions heretofore known suffer from a number of disadvantageswhich include but not are not limited to one or more of being difficultto use, taking too long, having poor quality matching, not being highlypredictive of success on the job, failing to measure what matters, notsaving time, not being reliable, not being customized, not beingcustomized for a specific local job, requiring too much guess-work inthe hiring process, slot being focused, not being statisticallyvalid/sound, being difficult to deploy on a massive scale, not beingautomatic/automated, being difficult to deploy across a large number ofcandidates, being limited to practical use by large scale employersonly, failing to bring rigor and/or discipline to the hiring process,not being user friendly, not being standardized, and not being legallydefensible.

What is needed is an assessment system and method that solves one ormore of the problems described herein and/or one or more problems thatmay come to the attention of one skilled in the art upon becomingfamiliar with this specification.

SUMMARY OF THE INVENTION

The present invention has been developed in response to the presentstate of the art, and in particular, in response to the problems andneeds in the an that have not yet been fully solved by currentlyavailable assessment systems and methods. Accordingly, the preset tinvention has been developed to provide an effective employmentassessment system and method including a processor over a computerizednetwork.

According to one embodiment of the invention, there is an assessmentsystem configured to automatically assess an individual and anassociated job using a computing device. The system may include abenchmark module that may store a benchmark associated with a job. Thebenchmark may include both minimum threshold scores and maximumthreshold scores associated with particular characteristics. The systemmay it an electronic library of questions associated with an array ofcharacteristics.

The system may include a survey module that may be functionally coupledto the electronic library and the benchmark module that mayautomatically perform an assessment by automatically selecting questionsfrom the electronic library based on the benchmark and applying, suchquestions to an individual using a user interface module. The surveymodule may automatically generate a benchmark associated with a job byapplying an assessment to a pool of individuals that perform that job.

The system may include an analysis module that may be functionallycoupled to the survey module that analyzes application of the assessmentprocess to the individual. The analysis module may automaticallygenerate a report that details the analysis of the individual withrespect to the job. The assessment system may include a benchmarklibrary that may have electronic records of an array of benchmarksagainst which the analysis module may compare an individual Theassessment system may include an update module that may modify thebenchmark based on real-time employment data from an employer.

According to one embodiment of the invention, there is a method ofautomatically assessing an individual and an associated job using acomputing device. The method may include the step of storing a benchmarkassociated with a job. The method may include the step of associating abenchmark with a plurality of characteristics. The method may includethe step of providing an electronic library of questions wherein thequestions are associated with an array of characteristics including theplurality of characteristics that may be specifically associated with ajob analysis that may be locally validated.

The method of automatically assessing an individual and an associatedjob using a computing device may include the step of automaticallyselecting questions from the electronic library of questions based onthe benchmark and its associated plurality of characteristics. Themethod may include the step of applying the automatically selectedquestions to an individual. The method may include the step ofautomatically analyzing application of the assessment process to theindividual using a computing device.

The method of automatically assessing an individual and an associatedjob using a computing device may include the step of automaticallygenerating a benchmark associated with a job by applying an assessmentto a pool of individuals that perform that job. The method may includethe step of generating both minimum threshold scores and maximumthreshold scores associated with the plurality of characteristics whileassociating the benchmark with the plurality of characteristics. Themethod may include the step of providing a benchmark library includingelectronic records of an array of benchmarks against which the analysismodule may compare an individual and automatically applyingautomatically selected questions associated with each of the array ofbenchmarks to the individual.

The method of automatically assessing an individual and an associatedjob using a computing device may include the step of automaticallygenerates a report that details the analysis of the individual withrespect to the job. The method may include the step of modifying thebenchmark based on real-time employment data from an employer, includingreal-time data regarding hiring and firing of employees within anemployee pool associated with a particular benchmark. The method mayinclude the step of automatically generates a benchmark associated witha job by applying an assessment to a pool of individuals that performthat job.

The method of automatically assessing an individual and an associatedjob using a computing device may include the step of generating bothminimum threshold scores and maximum threshold scores associated withthe plurality of characteristics while associating the benchmark withthe plurality of characteristics. The method may include the step ofproviding a benchmark library including electronic records of an arrayof benchmarks against which the analysis module may compare anindividual and automatically applying automatically selected questionsassociated with each of the array of benchmarks to the individual. Themethod may include the step of automatically generating a report thatdetails the analysis of the individual with respect to the job. Themethod may include the step of modifying the benchmark based onreal-time employment data from an employer, including real-time dataregarding, hiring and firing of employees within an employee poolassociated with a particular benchmark.

Reference throughout this specification to features, advantages, orsimilar language does not imply that all of the features and advantagesthat may be realized with the present invention should be or are in anysingle embodiment of the invention. Rather, language referring to thefeatures and advantages is understood to mean that a specific feature,advantage, or characteristic described in connection with an embodimentis included in at least one embodiment of the present invention. Thus,discussion of the features and advantages, and similar language,throughout this specification may, but do not necessarily, refer to thesame embodiment.

Furthermore, the described features, advantages, and characteristics ofthe invention may be combined in any suitable manner in one or moreembodiments. One skilled in the relevant art will recognize that theinvention can be practiced without one or more of the specific featuresor advantages of a particular embodiment. In other instances, additionalfeatures and advantages may be recognized in certain embodiments thatmay not be present in all embodiments of the invention.

These features and advantages of the present invention will become morefully apparent from the following description and appended claims, ormay be learned by the practice of the invention as set forthhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

In order for the advantages of the invention to be readily understood, amore particular description of the invention briefly described abovewill be rendered by reference to specific embodiments that areillustrated in the appended drawing(s). It is noted that the drawings ofthe invention are not to scale. The drawings are mere schematicsrepresentations, not intended to portray specific parameters of theinvention. Understanding that these drawing(s) depict only typicalembodiments of the invention and are not, therefore, to be considered tobe limiting its scope, the invention will be described and explainedwith additional specificity and detail through the use of theaccompanying, drawing(s), in which:

FIG. 1 is a network diagram of an assessment system over a computerizednetwork, according to one embodiment of the invention;

FIG. 2 is a module diagram of an assessment system, according to oneembodiment of the invention;

FIG. 3 is a flowchart of a method of automatically assessing anindividual and an associated job using a computing device, according toone embodiment of the invention;

FIG. 4 is a sequence diagram of a method of automatically assessing anindividual and an associated job using a computing device, according toone embodiment of the invention;

FIG. 5 is a chart illustrating a set of non-limiting exemplary methodsof automatically, benchmarking and scoring of an assessment system,according to various embodiment of the invention; and

FIG. 6 is a chart illustrating a set of non limiting exemplary methodsof automatically benchmarking and scoring of an assessment system,according to various embodiment of the invention.

DETAILED DESCRIPTION OF THE INVENTION

For the purposes of promoting an understanding of the principles of theinvention, reference will now be made to the exemplary embodimentsillustrated in the drawing(s), and specific language will be used todescribe the same. It will nevertheless be understood that no limitationof the scope of the invention is thereby intended. Any alterations andfurther modifications of the inventive features illustrated herein, andany additional applications of the principles of the invention asillustrated herein, which would occur to one skilled in the relevant artand having possession of this disclosure, are to be considered withinthe scope of the invention.

Many of the functional units described in this specification have beenlabeled as modules in order to more particularly emphasize theirimplementation independence. For example, a module may be implemented asa hardware circuit comprising custom VLSI circuits or gate arrays,off-the-shelf semiconductors such as logic chips, transistors, or otherdiscrete components. A module may also be implemented in programmablehardware devices such as field programmable gate arrays, programmablearray logic, programmable logic devices or the like. Modules may also beimplemented in software for execution by various types of processors. Anidentified module of programmable or executable code may, for instance,comprise one or more physical or logical blocks of computer instructionswhich may, for instance, be organized as an object, procedure, orfunction.

Nevertheless, the executables of an identified module need not bephysically located together, but may comprise disparate instructionsstored in different locations which, when joined logically together,comprise the module and achieve the stated purpose for the module.Indeed, a module and/or a program of executable code may be a singleinstruction, or many instructions, and may even be distributed overseveral different code segments, among different programs, and acrossseveral memory devices. Similarly, operational data may be identifiedand illustrated herein within modules, and may be embodied in anysuitable form and organized within any suitable type of data structure.The operational data may be collected as a single data set, or may bedistributed over different locations including over different storagedevices, and may exist, at least partially, merely as electronic signalson a system or network.

The various system components and/or modules discussed herein mayinclude one or more of the following: a host server, motherboard,network, chipset or other computing system including a processor forprocessing digital data; a memory device coupled to a processor forstoring digital data; an input digitizer coupled to a processor forinputting digital data; an application program stored in a memory deviceand accessible by a processor for directing processing of digital databy the processor; a display device coupled to a processor and/or amemory device for displaying information derived from digital dataprocessed by the processor; and a plurality of databases includingmemory device(s) and/or hardware/software driven logical data storagestructure(s).

Various databases/memory devices described herein may include recordsassociated with one or more functions, purposes, intended beneficiaries,benefits and the like of one or more modules as described herein or asone of ordinary skill in the art would recognize as appropriate and/orlike data useful in the operation of the present invention.

As those skilled in the art will appreciate, any computers discussedherein may include an operating system, such as but not limited to:Android, iOS, BSD, IBM z/OS, Windows Phone, Windows CE, Palm OS, WindowsVista, NT, 95/98/2000, OS X, OS2; QNX, UNIX; GNU/Linux, Solaris; MacOS;and etc., as well as various conventional support software and driverstypically associated with computers. The computers may be in a home,industrial or business environment with access to a network. In anexemplary embodiment, access is through the Internet through acommercially-available web-browser software package, including but notlimited to Internet Explorer, Google Chrome, Firefox, Opera, and Safari.

The present invention may be described herein in terms of functionalblock components, functions, options, screen shots, user interactions,optional selections, various processing steps, features, userinterfaces, and the like. Each of such described herein may be one ormore modules in exemplary embodiments of the invention even if notexpressly named herein as being a module. It should be appreciated thatsuch functional blocks and etc. may be realized by any number ofhardware and/or software components configured to perform the specifiedfunctions. For example, the present invention may employ variousintegrated circuit components, e.g., memory elements, processingelements, logic elements, scripts, look-up tables, and the like, whichmay carry out a variety of functions under the control of one or moremicroprocessors or other control devices. Similarly, the softwareelements of the present invention may be implemented with anyprogramming or scripting language such as but not limited to Eiffel,Haskell, C, C++, Java, Python, COBOL, Ruby, assembler, Groovy, PERL,Ada, Visual Basic, SQL Stored Procedures, AJAX, Bean Shell, andextensible markup language (XML), with the various algorithms beingimplemented with any combination of data structures, objects, processes,routines or titer programming elements. Further, it should be noted thatthe present invention may employ any number of conventional techniquesfor data transmission, signaling, data processing, network control, andthe like. Still further, the invention may detect or prevent securityissues with a client-side scripting language, such as JavaScript,VBScript or the like.

Additionally, many of the functional units and/or modules herein aredescribed as being “in communication” with other functional units, thirdparty devices/systems and/or modules. Being, “in communication” refersto any manner and/or way in which functional units and/or modules, suchas, but not limited to, computers, networks, mobile devices, programblocks, chips, scripts, drivers, instruction sets, databases and othertypes of hardware and/or software, may be in communication with eachother. Some non-limiting examples include communicating, sending, and/orreceiving data and metadata via: a wired network, a wireless network,shared access databases, circuitry, phone lines, internet backbones,transponders, network cards, busses, satellite signals, electricsignals, electrical and magnetic fields and/or pulses, and/or so forth.

As used herein, the term “network” includes any electroniccommunications means which incorporates both hardware and softwarecomponents of such. Communication among the parties in accordance withthe present invention may be accomplished through any suitablecommunication channels, such as, for example, a telephone network, anextranet, an Intranet, Internet, point of interaction device (point ofsale device, personal digital assistant, cellular phone, kiosk, etc.),online communications, off-line communications, wireless communications,transponder communications, local area network (LAN), wide area network(WAN), networked or linked devices and/or the like. Moreover, althoughthe invention may be implemented with TCP/IP communications protocols,the invention may also be implemented using other protocols, includingbut not limited to IPX, Appletalk, IP-6, NetBIOS, OSI or any number refexisting or future protocols. If the network is in the nature of apublic network, such as the Internet, it may be advantageous to presumethe network to be insecure and open to eavesdroppers. Specificinformation related to the protocols, standards, and applicationsoftware utilized in connection with the Internet is generally known tothose skilled in the art and, as such, creed not be detailed herein.See, for example, DILIP NAIK, INTERNET STANDARDS AND PROTOCOLS (1998);JAVA 2 COMPLETE, various authors, (Sybex 1999); DEBORAH RAY AND ERICRAY, MASTERING HTML 4.0 (1997); and LOSHIN, TCP/IP CLEARLY EXPLAINED(1997), the contents of which are hereby incorporated by reference.

Reference throughout this specification to an “embodiment,” an “example”or similar language means that a particular feature, structure,characteristic, or combinations thereof described in connection with theembodiment is included in at least one embodiment of the presentinvention. Thus, appearances of the phrases an “embodiment,” an“example,” and similar language throughout this specification may, butdo not necessarily, all refer to the same embodiment, to differentembodiments, or to one or more of the figures. Additionally, referenceto the wording “embodiment,” “example” or the like, for two or morefeatures, elements, etc. does not mean that the features are necessarilyrelated, dissimilar, the same, etc.

Each statement of an embodiment, or example, is to be consideredindependent of any other statement of an embodiment despite any use ofsimilar or identical language characterizing each embodiment. Therefore,where one embodiment is identified as “another embodiment,” theidentified embodiment is independent of any other embodimentscharacterized by the language “another embodiment.” The features,functions, and the like described herein are considered to be able to becombined in whole or in part one with another as the claims and/or artmay direct, either directly or indirectly, implicitly or explicitly.

As used herein, “comprising,” “including,” “containing,” “is,” “are,”“characterized by,” and grammatical equivalents thereof are inclusive oropen-ended terms that do not exclude additional unrecited elements ormethod steps. “Comprising” is to be interpreted as including the morerestrictive terms “consisting of” and “consisting essentially of.”

FIG. 1 is a network diagram of an assessment system over a computerizednetwork, according to one embodiment of the invention. Such a system maybe used by an employer to find quality candidates and/or by aprospective employee to find quality job opportunities within anorganization or among a set of organizations.

There is shown as assessment system 10 in communication with a userinterface module(s) 12 and an employer interface module(s) 14 over acomputerized network 16. The assessment system 10 communicates with eachof the user interface module(s) 12 and the employer interface module(s)14 and automatically assesses prospective employees over the userinterface module(s) 12 and repots/coordinates the processing of the samewith employers over the employer interface module(s) 14. As anon-limiting example, such may be distributed over the internet, withthe assessment system 10 being a server connected thereto and the userinterface module(s) 12 and the employer interface module(s) 14 beingpersonal computers and/or smartphone connected thereto throughbrowsers/apps that may be resident on such devices and that may connectto application services and or one or mi re websites served by theassessment system server. The illustrated employer interface module 14may include one or more modules configured for use by a prospectiveemployee to automatically assesses prospective employers/jobs over theuser interface module(s) 12 and/r to report/coordinate the processing ofthe same over the employer interface module(s) 14.

The illustrated assessment system 10 is designed to assistemployers/employees in finding the most qualified candidates/positions.Such may be used for other purposes and/or in other contexts, includingbut not limited to assessing suitability of volunteers with eventsand/or charitable causes; mentors with mentees; corporate or charityboard members with board positions; vendors with clients; and the likeand/or or vice-versa for any of the contexts described herein. Such asystem may be deployable massively over a distributed network whereinthose being assessed use their own devices or those accessible to themas appropriate and needed in order to interface with the system, andthus is highly scalable for large employers but still accessible forsmall employers. Such a system may be utilized for career planning, forindividuals and/or organizations; educational streaming/screening (e.g.by individuals, by post-secondary institutions); and/or may be utilizedby associations and/or not-for-profits for purposes similar to thosedescribed herein. Such a system may be integrated and/or functionallycoupled to one or more automated job boards, job syndicators,operational systems of temp/placement agencies, HR systems, and/orapplicant tracking systems.

The assessment system 10 uses specific questions that are automaticallyselected by the system as being relevant to a specific employmentposition as a benchmark to filter for and find the most qualifiedcandidates. The benchmark may be set by the employer seeking candidatesthrough an employer interface module 14 in communication with the systemover a computerized network 16.

The benchmark may be set automatically following an automated interviewprocess and job analysis with the employer about the position and/or itmay be set and/or modified by historical data regarding the positionfrom the employer (or otherwise) such as but not limited to data enteredfront a study performed against past employees having that positionand/or through an automated and/or continually/regularly/systematicallyupdating interface with the HR records of the employer, therebyimproving the benchmark correlation with success in the position overtime. The system may automatically generate one or more custom reports,such as but not limited to an interview guide that may be customized toa job/position, may be customized to a particular applicant and/or maybe customized to the specific interplay between a job/position and aspecific applicant. As such, it may be that different interview guidesare automatically generated for different applicants (and/or for thesame applicant applying to two different jobs) and that such interviewguides may differ in content, sequence of information, questions to ask,suggested follow-up questions to ask, and the like and combinationsthereof.

The illustrated assessment system 10 is in communication with a userinterface module 12 or a plurality of user interface modules over acomputerized network 46 to complete the employment selection process.The employer may use the assessment system 10 to set standards,characteristics, traits, etc. which would lead, hypothetically, tosuccess at the open employment position. The employment candidate usesthe assessment system 10 through the user interface module 12 over acomputerized network 16 to complete questionnaires for one or moreemployment positions.

According to one embodiment of the invention, there is an assessmentsystem 10 designed to customize a benchmark and a test/questionnairethat employment candidates take for an employment position. Thebenchmark drives the automated customization of the test/questionnaireby the system. The benchmark may be determined as a result of a jobanalysis (e.g. a survey(s) completed by someone familiar with the job orcompleted by those holding the jobs, identifying behaviors andpersonality traits associated with successful holders of that jobposition). The assessment system 10 automatically analyzes the resultsof the survey(s) and thereby selects from a large set of behavioralcharacteristics to identify those most closely correlated with successand the ranges within them (not just the highest scores) that are mostclosely correlated with success.

The system 10 then automatically generates a test/questionnairecustomized to the results of the survey analysis and automaticallygenerates a grading/selecting/filtering process according to suchresults. Automatic generation of a test/questionnaire may be by script,function, algorithm or otherwise wherein the system, based on particularcriteria, chooses questions from a library of questions wherein suchquestions are tagged or otherwise associated with particularcharacteristics associated with characteristics sought for with regardto a particular position/job.

The system 10 automatically generates a “FitScore” between employmentapplicant's test results and the benchmark and then reports the same tothe employer. Follow-up questions/processes may be automaticallygenerated and communicated based on test results and analysis. Thesystem 10 also matches candidates against multiple job listings thathave benchmarks associated with them and their best matches areautomatically reported to the employer.

The assessment system 10 is configured to provide an easier, shorter,better matching, highly predictive of success in the job, measures whatmatters, saves time, job-specific, scientifically validated, morereliable, customized, removes guesswork from the hiring process, morefocused, statistically valid, easy to deploy on a massive scale,automatic, easy to deploy across a large number of candidates, notlimited to large scale employers, brings rigor and discipline to thehiring process, user friendly but still rigorous, highly standardizedacross applicants for the same position, norm-referenced (e.g. scoringmay be in reference to a normal distribution of a population), andlegally defensible employment assessment system and method.

Advantageously, the system 10 may allow for the automated generation ofuseful benchmarks against which applicants may be measured. Further, thesystem may automatically generate assessment tools based on thebenchmarks and customized thereto, thereby not including assessmentquestions/tools uncorrelated with successful applicants. The system 10may be massively deployed over a network to be used by many employers ofmany different types and sizes who are filtering through any number ofapplicants and allow such employers to usefully filter down tomanageable numbers of candidates from any candidate population size. Thesystem 10 may automatically provide additional features and benefits forthe applicants and/or the employers during and after the assessmentprocess that may additionally support the efforts of such parties,including but not limited to redirecting applicants to differentpositions/employers for which they may be better suited and/orgenerating useful follow-up assessment tools/ideas for employers to usein further filtering among applicants.

FIG. 2 is a module diagram of an assessment system, according to oneembodiment of the invention. There is shown an assessment system 10including a control module 32, a communication module 34, a data storagemodule 36, an analysis module 26, a benchmark module 20, a survey module24, an electronic library of questions 22, a benchmark library ofelectronic records 28, and a update module 30 each in communication witheach other as appropriate to perform their various functions. Othermodules not shown may be present and/or may be incorporated within theillustrated modules, including but not limited to an administrationmodule, a scoring module, and a reporting module. The illustratedmodules operate together to provide automated benchmark generation andanalysis, automated customization of assessment tools based on suchbenchmarks, deployment of the same to a multiplicity of candidates,automated analysis of such assessments as measured against thebenchmarks, and ancillary features and functions as described herein.

The illustrated assessment system 10 automatically assess an individualand an associated job using a computing device. The system 10 includes abenchmark module 20 that stores a benchmark associated with a job. Thebenchmark may include both minimum threshold scores and maximumthreshold scores associated with one or more particular characteristics.As a non-limiting example, a benchmark for a supporting position mayinclude a maximum threshold score for an aggressiveness-typecharacteristic/trait that may be less than a maximum allowable score,while a benchmark for a job requiring meticulous attention to detail mayinclude a maximum threshold score for an creativity-typecharacteristic/trait that may be less than a maximum allowable score.The benchmark may be as simple as record of characteristics, each withminimum, maximum, and/or target scores/values. Benchmarks may be morecomplicated wherein cross-characteristic patterns/dependencies bedesired (e.g. high in a particular characteristic is desired unlessanother particular characteristic is also high) or wherein suchpatterns/dependencies may have been observed from successful employeesin such jobs.

The system 10 includes an electronic library of questions 22 associatedwith an array of characteristics. The questions may be tagged orotherwise associated with particular traits/characteristics such thatthe system can draw from those questions as needed to automaticallygenerate tests/etc. using such questions. The questions may also includeother tags/markers for use by the system to generate better tests, suchas but not limited to question groupings (e.g. the system pulls allquestions from a particular grouping (e.g. questions grouped byeducation level to understand the question) or the system is restrictedfrom pulling a questions from the same grouping twice (e.g. questionsgrouped in order to maxi system's ability to detect when the test takeris defrauding the test)). The system may utilize such tags in automatedgeneration of tests according to benchmark(s). The electronic librarymay also include tags associated with intended use or expected use ofthe questions, especially in the case where a single system servicesmultiple user group types. Such may include but it not limited tomarking a question as suitable for employment tests, while otherquestions are suitable for non-employment situations.

The illustrated system 10 includes a survey module 24 functionallycoupled to the electronic library 22 and the benchmark module 20 toautomatically perform an assessment by automatically selecting questionsfrom the electronic library 22 based on the benchmark and applying suchquestions to an individual using a user interface module. The surveymodule 24 may also automatically generate a benchmark associated with ajob by applying an assessment to a pool of individuals that perform thatjob and recording the results in association with data regarding thoseindividuals performance at that job.

The illustrated assessment system 10 includes an analysis module 26functionally coupled to the survey module 24 that analyzes applicationof the assessment process to the individual. The analysis module 26automatically generates a report that details the analysis of theindividual with respect to the job. The assessment system 10 includes abenchmark library 28 having electronic records of an array of benchmarksagainst which the analysis module 26 compares an individual.Accordingly, a prospective employee may take a test that includesquestions associated with all characteristics relevant to a set ofbenchmarks and then the results of the test may be analyzed by theassessment system and a report may be generated that, instead ofcomparing a plurality of candidates with a single job, compares a singlecandidate with a plurality of jobs. The assessment system 10 includes anupdate module 30 modifies the benchmark based on real-time employmentdata from an employer.

The illustrated assessment system 10 includes a control module 32 thatprovides operational instructions and cot =ands to the modules andcomponents of the system 10. The control module 32 is in communicationwith the modules and components of the system 10 (and/or other modulesdescribed herein) and provides managerial instructions and commandsthereto. The source of such instructions/commands may be from one ormore other modules described herein and/or through interactions betweenone or more other modules described herein. The control module 32 setsparameters and settings for each module and component of the system 10.Non-limiting examples of a control module may be a control moduledescribed in U.S. Pat. No. 5,430,836, issued to Wolf et al.; or acontrol module described in U.S. Pat. No. 6,243,635, issued to Swan etal. which are incorporated for their supporting teachings herein. Acontrol module may include but is not limited to a processor, a statemachine, a script, a decision tree, and the like.

The illustrated assessment system 20 includes a communication module 34,such as a network card, system bus, or wireless communication module,and communicates with a computerized network. The communication module34 provides communication capabilities, such as wireless communication,to the modules and components of the system 10 and the components andother modules described herein. The communication module 34 providescommunication between a wireless device, such as a mobile phone, and acomputerized network and/or to facilitate communication between a mobiledevice and other modules described herein. The communication module 34may have a component thereof that is resident on a user's mobile deviceor on a user's desktop computer. Non-limiting examples of acommunication module may be but not limited to: a communication moduledescribed in U.S. Pat. No. 5,307,463, issued to Hyatt et al.; or acommunication module described in U.S. Pat. No. 6,133,886, issued toFariello et al., which are incorporated for their supported herein.

The illustrated system 10 includes a data storage module 36 incommunication with the modules and components of the system 10. The datastorage module 36 collects and store data for each of the modules of thesystem 10. The data storage module 36 is in communication with thevarious modules and components of the system 10 over a computerizednetwork and stores data transferred there through. The data storagemodule 36 stores data transferred through each of the modules of thesystem 10, thereby updating the system with up to date data and realtime employment data. The data storage module 36 securely stores userdata along with data transferred through the system 10. Data storagemodules 36 may be databases and/or data files and the memory storagedevice may be, but is not limited to, hard drives, flash memory, opticaldiscs, RAM, ROM, and/or tapes. A non-limiting example of a data storagemodule may be a Filemaker Pro 11, manufactured by Filemaker Inc., 5261Patrick Henry Dr., Santa Clara, Calif., 95054. Non-limiting examples ofa data storage module may include: a HP Storage Works P2000 G3 ModularSmart Array System, manufactured by Hewlett-Packard Company, 3000Hanover Street, Palo Alto, Calif., 94304, USA; or a Sony Pocket Bit USBFlash Drive, manufactured by Sony Corporation of America, 550 MadisonAvenue, New York, N.Y., 10022.

The illustrated assessment system 10 includes an analysis module 26 thatincludes executable information regarding employment positions,characteristics, qualifications, traits, etc. and user data. Theanalysis module 26 is in communication with the modules and componentsof the system 10 as appropriate to perform its function(s). The analysismodule 26 analyzes collections of employment data according to theexecutable information regarding employment positions and employmentcandidates, such as metadata. The analysis module 26 analyzes storedmetadata from the data storage module 36 thereby generating a rating orsummary tailored for a specific employment position. The analysis module26 is configured to analyze user employment data from a set of questionscustomized by the employer to determine which selection of employmentoptions best matches the candidate and the position, based on the user'sanswers to the questions from the system 10. The analysis module 26 mayprocess assessment data from candidates for scoring and/or forpreparation for scoring. Such may include comparing responses againsttarget responses and/or target response ranges, recording variance datafrom the same, notating large variances, validating response sets, errorchecking response sets, comparing against secondary, additional,ancillary, and/or even unrelated benchmarks and the like andcombinations thereof. Such may be performed automatically using scripts,functions, matching engines, data conditioning modules, data processingmodules and the like and combinations thereof. Non-limiting examples ofanalysis module may be a data analysis system as described in U.S.Patent Publication No.: 2012/0290576; or an analysis system as describedin U.S. Patent Publication No.: 2011/0208519, which are incorporated fortheir supporting teachings herein. Non-limiting examples of a knowledgebase module may be as described in U.S. Pat. No. 6,064,971 by Hartnettand U.S. Pat. No. 5,257,185 by Farley, which are incorporated for theirsupporting teachings herein.

The illustrated assessment system 10 includes a survey module 24 (and/ora study module) that is in communication with the modules and componentsof the system 10 and that gathers/manages/uses performance and/orbehavioral data to help automatically generate benchmarks. The surveymodule 24 includes a study module which may gather/manage/use real datafrom existing employees to help automatically generate benchmarks. Thestudy module may perform and/or automatically perform acriterion-related study to determine the predictive validity of anassessment to job performance. The survey module 24 is in communicationwith an employer interface module over a computerized network, whereinthe employer uses the employer interface module to set standards,characteristics, traits, qualities, that would identify a qualified andsuccessful candidate for a particular employment position. The surveymodule 24 includes an application and/or website that may be served overa network and may include questions associated with building a benchmarkassociated with a position. The survey module 24 includes accountmanagement and access modules that allow employers to store suchinformation in association with their account and/or to manage aplurality of benchmarks in the form of different job positions that theyseek to fill. The survey module 24 may include a pattern library module,wherein sets of data, job descriptions, job postings, benchmarks, etc.that may be unique to each employer/etc. may be stored, managed and/orbe kept separate or otherwise secure. There may also be a common patternlibrary account that may store sets of data, job descriptions, jobpostings, benchmarks, etc. that may be accessible by one or more useraccounts, including but not limited to employer/etc. accounts.Accordingly, an employer could use common data to facilitate in thegeneration of private and unique job descriptions and benchmarksassociated therewith and then run data from applicants applying for oneparticular job in the set against a different job within the set (e.g.to explore fits for career/succession planning). Such modules may alsobe in communication with one or more data management modules of theemployer that may feed information to the system regarding informationrelevant to building and/or modifying benchmarks, such as but notlimited to actual employment data (e.g. duration of employment, resultsof performance evaluations). Non-limiting examples of asurvey/concurrent study module may lie as described in U.S. PatentPublication No.: 2011/0000095 or a survey system as described in U.S.Pat. No. 8,635,099; which are both incorporated for their supportingteachings herein.

The illustrated assessment system 10 includes a benchmark module 20 thatis in communication with the modules and components of the system 10.The benchmark module 20 is in communication with the survey module 24,wherein the benchmark module 20 uses the data from the survey module 24to automatically generate a benchmark for a particular employmentposition. The benchmark may be used over and over again for a particularemployment position for a particular company or employer. The benchmarkwould be the standard as to how well the candidate would be likely toperform at the particular employment position. As a non-limitingexample, such may be accomplished by scoring/weighting results from realdata and/or survey(s) regarding the position from one or more sources(e.g. libraries, survey results, study results, reports, etc.) todetermine which characteristics, traits, skills, abilities, interests,references, desires, past work history, ambitions, handwriting,demographics, psychological assessments, tendencies, types, etc. aremost closely associated with success in a position and to determine theranges of responses within such that are also most closely associatedwith success. Generally it is advisable for such a system to includesufficient questions, questions types, variations on the same question,and the like in order to validate the responses and cross-check thevalidity of the resulting analysis so that, any benchmarks derivedtherefrom are more likely to be useful during the assessment process.Non-limiting examples of a benchmark module may be a system as describedin U.S. Pat. No. 6,873,934, or a benchmark module as described in U.S.Patent Publication No.: 2014/0207415; which are both incorporated fortheir supporting teachings herein.

The illustrated assessment system 10 includes an electronic library ofquestions 22 associated with an array of characteristics. The electroniclibrary 22 is in communication with the modules and components of thesystem 10, wherein the electronic library 22 is in communication withthe data storage module 36 to store a plurality of questions for anemployer to select from, when looking to hire an new employee or whenlooking to promote or move current employees within the company.Non-limiting examples of an electronic library of questions may be anelectronic library as described in U.S. Pat. No. 8,661,185, issued toThorp et al.; or a library system as described in U.S. Pat. No.7,287,214, issued to Jenkins et al., which are both incorporated fortheir supporting teachings herein.

The illustrated assessment system 10 includes a benchmark library 28including electronic records of an array of benchmarks against which theanalysis module 26 may compare an individual to find an employmentposition that best suits the individual. The benchmark library 28 is incommunication With the modules and components the system 10. Thebenchmark library 28 is in communication with the data storage module 36and the benchmark module 20, wherein the benchmark library storesbenchmarks and associated data therewith through the data storage module36 and receives a plurality of electronic records of an array ofbenchmarks from the benchmark module 20. Non-limiting examples of abenchmark library may be an electronic library as described in U.S. Pat.No. 8,661,185, issued to Thorp et al.; or a library system as describedin U.S. Pat. No. 7,287,214, issued to Jenkins et al., which are bothincorporated for their supporting teachings herein.

The illustrated assessment system 10 includes an update module 30 thatmodifies the benchmark based on real-time employment data from anemployer. The update module 30 is in communication with the modules andcomponents of the s stem 10. The update module is in communication withthe benchmark module 20 and an employer interface module, wherein theemployer may set new benchmarks and the update module 30 automaticallyupdates the employers array of benchmarks from the benchmark module 20in real-time. The update module 30 is in communication with the datastorage module to store newly modified array of benchmarks. Non-limitingexamples of an update module may be a method and module as described inU.S. Patent Publication No.: 2014/0306973, by Yang et al.; or an updatemodule as described in U.S. Patent Publication No.: 2010/0161219, by Lo;which are both incorporated for their supporting teachings herein.

There may be an assessment generation module (not illustrated) that mayautomatically generate one or more assessment tools (e.g.questionnaires, surveys, skill tests) based on characteristics, traits,skills, abilities, interests, references, desires, past work history,ambitions, handwriting, demographics, psychological assessments, etc.identified within the benchmark (and ranges thereof) as being relevant.The assessment generation module may automatically generate anassessment based on information from one or more sets of benchmarks. Theassessment generation module may automatically select from a library ofassessment tools, questions, exams, that may be associated with alibrary/table of traits, characteristics, skills, abilities, tendencies,etc. whereby the tools/etc. may be indexed and thus automaticallyselected by the assessment generation module. The assessment generationmodule may include scripts, boundary conditions, formatting requirementsand the like that may be define limits to what may be selected and howit may be presented. Such may be associated with one or more useraccounts such that different employers may have different assessmentswith different levels of rigor, duration, etc. An assessment generationmodule ma automatically include “noise” in the form of tools/etc. thatare not closely associated with success in order to reduce “gaming” thesystem or espionage acts. There may also be adaptive questions (e.g.questions that rule out portions of a range with respect to a trait butdo not determine an exact score on that trait) and/or assessment toolsthat include randomized question sets. As a non-limiting example, such amodule may automatically select questions from a library of questionsthat are indexed according to desired traits. Non-limiting examples ofan assessment generation module include modules described in US PatentApplication Nos. 20080208665 and 20050276396, which are bothincorporated by reference herein for their supporting teachings.

The assessment system may include a scale module, not shown, which is incommunication with the modules and components of the system. The scalemodule is in communication with the analysis module and the benchmarkmodule to generate a scale for a particular employment position withregard to one or more traits, characteristics, etc. of the same. Thescale module may set limitations, parameters, regulations for aparticular employment position. Such a scale may be within upper andlower bounds of a rating of a trait/etc. instead of merely beingrepresented as above or below a particular threshold. The scale moduleallows a user to identify where they are on the scale, and/or mayidentify where the candidate needs to improve to compete with otheremployment candidates in a particular employment field. Non-limitingexamples of a scale module may be a rating system as described in U.S.Patent Publication No.: 2012/0220364 or a system as described in U.S.Patent Publication No.: 2014/0200879; which are both incorporated fortheir supporting teachings herein.

The assessment system may include a scoring module, not shown, that isin communication with the modules and components of the system. Thescoring module is in communication with the scale module in order togenerate an overall score for a employment candidate for a particularemployment position. The scoring module is configured to set criteria,scoring parameters, ratings, etc. for a particular employment position.The scoring module then rates the answers from an employment candidatefor a particular employment position and gives them a score or rating.The scoring module may be static in that it scores a whole test,including portions not relevant to a particular benchmark. The scoringmodule adaptively scores such that traits/etc. that are weighted asbeing more important receive a higher degree of certainty in the scoringand others are scored less rigorously. Non-limiting examples of ascoring module may be a scoring module as described in U.S. PatentPublication No.: 2014/0236927; or a scoring system as described in U.S.Pat. No. 8,157,566; which are both incorporated for their supportingteachings herein.

The assessment system 10 may include a report module, not shown, incommunication with the data storage module. The report module isconfigured to automatically generate one or more reports, including butnot limited to employment analysis information front the scoring module.The report module is configured to set parameters, criteria,characteristics, settings, preferences, listings, categories, groupings,etc. for a report, for employment candidates to review how they ratedwith the other candidates. Non-limiting examples of reports and thesubject matter thereof are provided in Appendix A. Non-limiting examplesof a report module may be a system as described in U.S. Pat. No.7,711,581; or a report generation module as described in U.S. PatentPublication No.: 2012/0284188, which are incorporated for theirsupporting teachings herein.

According to one embodiment of the invention, there is an assessmentsystem that automatically customizes the assessment based on anautomatically generated benchmark that is determined by analyzing datato determine what characteristics are correlated to success on the job(“fit with job,” e.g. how long will they stay and how well will theydo). There is a survey module (e.g. surveying manager to see what bethinks is good) or a concurrent study module (e.g. using real data fromexisting employees), a benchmark generation module (automaticallygenerates the benchmark, which is a list of behavioral characteristicsand their associated ranges where success is expected to be correlated),assessment generation module (automatically generates an assessmentbased on the benchmark), scoring module (automatically generatesimplements a filter/scoring system for the customized assessment basedon the benchmark), and a reporting module (automatically reportsfiltered results to the employer). There may be a position librarymodule (stores information about positions including associatedbenchmarks and other data like survey results). The library maycategorize benchmarks by position, by location, by division, byleadership, etc.

According to one embodiment of the invention, there is an assessmentsystem which is essentially a bank of hundreds of psychometricbehavioral and cognitive items intended to measure an individual'scharacteristics for the purpose of determining the best fit foremployment in a specific position. Assessments administered forparticular jobs are customized based upon extensive locally validatedjob analysis procedures and analysis of traits of existing employees whohave been successful in the position. The items used are grouped into 25personality constructs, or scales, and 1 cognitive reasoning scale. Thespecific scales measured are selected based on the results of the jobanalyses. This unique scale selection process improves the efficiencyand relevance of each assessment and reduces administration time whencompared to assessment systems which subject candidates to examinationof all scales regardless of the correlation of the scales to the job inquestion.

According to one embodiment of the invention, the assessment systemincludes a cognitive reasoning section which also may or may not beadministered depending upon the findings of the job analysis procedures.The items used to measure cognitive reasoning are based upon extensiveresearch into the types of cognitive tasks individuals should be able tosuccessfully complete at various levels of ability. While cognitiveability has been shown to be a good predictor of success in a job, thereare certain positions for which ability level may be gleaned from othersources such as education level or other extant test scores. Thecognitive reasoning portion of the assessment is provided as an option,but may not be administered to all job candidates, especially if thosecandidates have other proof of cognitive ability level.

FIG. 3 is a flowchart of a method of automatically assessing anindividual and an associated job using a computing device, according toone embodiment of the invention. There is shown a method ofautomatically assessing an individual and an associated job using acomputing device 40 over a computerized network.

The illustrated method 40 includes the step of storing a benchmarkassociated with a job 42. As a non-limiting example, such may beaccomplished by saving a record in a data storage device, such as butnot limited to a hard drive of a server. The benchmark may be in a dataformat associated with a particular database storage system and mayinclude a plurality of characteristics and/or associated scores,thresholds, ranges, patterns, dependencies and the like and combinationsthereof, and/or may include job information such as but not limited tojob title, employer, position quantity, deadline to fill position(s),job requirements, job location, and the like and combinations thereof

The illustrated method 40 includes the step of associating a benchmarkwith a plurality of characteristics 44. As a non-limiting example, suchmay be accomplished by appending a benchmark with characteristicinformation such as but not limited to characteristics and/or associatedscores, thresholds, ranges, patterns, dependencies and the like andcombinations thereof.

The illustrated method 40 includes the step of providing an electroniclibrary of questions 46 wherein the questions are associated with an Wayof characteristics including the plurality of characteristics. As anon-limiting example, such may be accomplished by connecting apreexisting library of questions to a system over a network and/or bygenerating an electronic library having questions associated withcharacteristics that are relevant to a particular job.

The illustrated method 40 includes the step of automatically selectingquestions from the electronic library of questions based on thebenchmark and its associated plurality of characteristics 48. Such mayinclude automatically not selecting questions that are not associatedwith a characteristic shown to be relevant to a particular job. As anon-limiting example, such may be accomplished by filtering questionrecords by characteristic with zero or more other tags (e.g. group tags)and randomly selecting questions that remain when the filter is applied.

The illustrated method 40 includes the step of applying theautomatically selected questions to an individual 50. As a non-limitingexample, such may be accomplished by operation of a survey module,printing out a test and delivering it to the individual, surveying athird party with regards to the individual, or the like or combinationsthereof The application may be an internet-based assessment which may bewithin the automated system, and/or n ay be customized for each jobwithin a given organization.

Employers may establish a system account, create a job in their library,and/or answer a survey for that, ob. They may add other positions and/orcomplete additional surveys, as desired. The system may generate a oneor more URL(s) for each position that may be unique, which may be foruse in advertising the position to candidates. When a job seeker clickson the URL, they may be brought into the system. They may have theoption of attaching a résumé and answering online interview questions aspart of their submission; once that part of the process is complete,they may be invited to complete the custom assessment for the positionto which they are applying.

Generally, there is no time limit for the assessment, however mostpeople should be able to complete it within 60 minutes. Personalityitems may offer brief descriptions of behavioral tendencies and/orattributes which may be responded to on a 5-point Likert-type scaleasking respondents to identify the frequency at which they believe theydisplay the behavior or attribute. Cognitive reasoning items may beformatted with multiple-choice responses.

The illustrated method 40 includes the step of automatically analyzingapplication of the assessment process to the individual using acomputing device 52. As a non-limiting example, such may be accomplishedby operation of an assessment module or otherwise processing answeredtest questions associated with the individual in comparison with one ormore benchmarks. Such processing may track compliance and/ornon-compliance with a benchmark, may report on patterns, may calculateone or more properties based on the answered questions (e.g. inferring alikelihood of success for the candidate, suggesting specific steps totake to help the candidate be more successful in their job).

Each domain or scale in the assessment may scored separately (e.g. on apercentile scale which has been nonmed on the working population ofNorth America). Percentile scores may then be automatically compared toscore ranges identified in the benchmarking process. A gap may bereported if candidate's scores fall outside that range. The larger thegap, the further away from the desired score range the candidate is on aparticular scale. A proprietary algorithm may be used to calculate a FitScore which provides an overall look at how closely a candidate matchesthe overall benchmark.

The illustrated method 40 includes the step of automatically generatinga benchmark associated with a job by applying an assessment to a pool ofindividuals that perform that job 54. Such may include eliminating oneor more characteristics .from a set of characteristics as being relevantto job performance. As a non-limiting example, such may be accomplishedby operation of a survey module such that the pool of individuals takesa broad spectrum test including questions associated with a multiplicityof characteristics such that patterns associated with the same may laterbe analyzed/Observed. Operation of a survey module may include but isnot limited to: pushing a test to a plurality of remote computingdevices (e.g. smart phones, tablet, desktop computers), printing a testand providing a machine readable user interface on which answers may beplaced (e.g. machine readable paper sheets, desktop computer,smartphone), and the like and combinations thereof.

A key to the effectiveness of any assessment system used for selectionand/or placement is local validation. That is, the assessment andresulting information should be relevant to the specific job for whichthe system is being used. To accomplish this, the system may use acomprehensive survey which identifies relevant scales for benchmarkingbut also identifies the scales which should be included in an assessmentfor a particular job, along with concurrent study capabilities toidentify the behavioral and cognitive factors which most greatlyinfluence the potential success of a candidate in a specific job. Whileother assessment systems require testees to respond to all questions inall stales regardless of their relevance to the job in question, thesystem may only require candidates to respond to items in those scaleswhich have proven to be associated with success on the job based on theresults of the position analysis functions.

Along with determining which scales need to be assessed for a specificjob, the survey may also automatically establish a range of scoreswithin those scales which have shown to be correlated with success onthe job. This may be accomplished by having a person familiar with thedemands of the job complete the survey, which contains an algorithm fordetermining score ranges, or by assessing incumbents in the job anddetermining the range of scores most often reported for those who are orhave been successful in the position. The score ranges for the scalesassessed become the benchmark against which all future candidates willbe compared to help determine their potential for successful performancein the job.

The illustrated method 40 includes the step of generating both minimumthreshold scores and maximum threshold scores associated with theplurality of characteristics while associating the benchmark with theplurality of characteristics 56. As a non-limiting example, such may beaccomplished by analyzing test results for a pool of job holders for aparticular job and observing (automatically or otherwise) characteristicpatterns among those who are the most/least successful at their jobs orwho have been so identified to the system, such as but not limited to byannotating a record regarding that employee in the system, and/or bysurveying an employer regarding desired characteristics for a particularjob.

The illustrated method 40 includes the stet of providing a benchmarklibrary including electronic records of an array of benchmarks againstwhich the analysis module may compare an individual and automaticallyapplying automatically selected questions associated with each of thearray of benchmarks to the individual 58. As a non-limiting example,such may be accomplished by functionally coupling a database storingsuch to the system, or generating such a database within the system.

The illustrated method of automatically assessing an individual and anassociated job using a computing device 40 includes the step ofautomatically generating a report that details the analysis of theindividual with respect to the job 60. As a non-limiting example, suchmay be accomplished by operating of a script that automatically reportsspecific information based on how well or how poorly test results matchup with a benchmark on a characteristic-by-characteristic basis and/orbased on specific patterns of characteristics.

The illustrated method 40 includes the step of modifying the benchmarkbased can real-time employment data from an employer, includingreal-time data regarding hiring and firing of employees within anemployee pool associated with a particular benchmark 62. As anon-limiting example, such may be accomplished by functionally couplingthe system to a system fur onboarding new hires and/or for handling exitprocessing of employees, such that data regarding performance, relevantcharacteristics, and the like may be appended to a job record and/or itsassociated benchmark(s) in real-time.

The illustrated method 40 includes the step of automatically generatinga benchmark associated with a job by applying an assessment to a pool ofindividuals that perform that job. As a on-limiting example such may beaccomplished by operation of a survey module and/or an assessment modulein as described herein.

The illustrated method of automatically assessing an individual and anassociated job using a computing device 40 includes the step ofgenerating both minimum threshold scores and maximum threshold scoresassociated with the plurality of characteristics while associating thebenchmark with the plurality of characteristics 64. As a non-limitingexample, such may be accomplished by operation of a survey module and/oran assessment module as described herein.

The illustrated method 40 includes the step of providing a benchmarklibrary including electronic records of an array of benchmarks againstwhich the analysis module may compare an individual and automaticallyapplying automatically selected questions associated with each of thearray of benchmarks to the individual 66. As a non-limiting example,such may be accomplished by operation of a survey module and/or anassessment module as described herein.

FIG. 4 is a sequence diagram of a method of automatically assessing anindividual and an associated job using a computing device, according toone embodiment of the invention. There is shown a user interface module12, an assessment system 10, and an employer interface module 14 incommunication with each other over a computerized network. Actionsand/or communications between the modules/systems are illustrated intime order (time progressing top to bottom) as labeled arrows betweenstems under the modules.

The illustrated user interface module 12 is in communication with anassessment system 10 and the assessment system 10 is in communicationwith an employer interface module 14. The assessment system 10 sends asurvey to the employer interface module 14, wherein the employerinterface module 14 fills out characteristics and qualities that wouldlead to a successful employee at their company. The employer interfacemodule 14 sends back the survey response to the assessment system 10,wherein the assessment system 10 evaluates the survey and determines thecriteria needed to find a successful employee that matches theemployer's needs.

The illustrated assessment system 10 sends a test to a plurality of userinterface modules 12, wherein the user interface module 12 completes thetest and sends back a test response assessment system 10. The assessmentsystem 10 analyzes the test responses from the user interface modules 12and matches all candidates against a benchmark for the intendedposition, and thereby identities those with the highest probability ofsuccess for the intended position.

The assessment system 10 sends test analysis and employee matches to theemployer interface module 14 for the employer to decide on whichcandidate to select for the employment position. The employer interfacemodule 14 sends employment selection to the assessment system 10,wherein the assessment system 10 sends employment results to the userinterface module 12. The user interface module 12 reviews the resultsfrom the assessment system 10 and responds with an employment agreement.The assessment system 10 notifies the employer interface module 14 thatthe user interface module 12 has accepted an employment position withthem.

Looking to FIGS. 5 and 6, an individual's score may be first calculatedby totaling the 1 to 5 ratings across all questions for eachscale/trait. The total raw scale score is then compared to a percentilelookup table derived from the pilot-testing of hundreds of subjects whoresponded to all items for all scales. The lookup table generates apercentile rank for each scale raw score and uses that percentile rank(1-100) as the score for all calculations and derivations throughout theprocess.

The traits or scales which are most relevant to a particular job aredetermined through the benchmarking process. The system uses resultsfrom the Position Analysis Survey and/or current employee results fromthe full assessment instrument in a criterion-related validity processwhich identifies those scales or traits which best differentiate highlyrated employees from those with lower performance metrics. Once thosetraits are identified, subsequent assessment instruments for thatparticular position contain only those scales which showed to be themost effective at identifying those individuals who have been successfulin the job. The system is pre-set to select only those scales with thetightest score groupings of top and bottom performers. For example,looking specifically at FIG. 5:

Example 1 shows the score distribution of 20 current employees (10 highperformers and 10 low performers) on a particular scale or trait, sayExtraversion. By virtue of the fact that all the top performers scoredin the lower end of the scale and all the low performers scored in thehigh end, the system would identity this as a relevant scale and set theresulting benchmark at score-range 1-3. This scale would then beselected for inclusion in subsequent assessments for applicants for thejob in question.

Example 2 shows the same 20 current employees on a different scale (sayAmbition) and the distribution of their scores. As can be seen by thewide spread and similar distribution of scores, this scale is not goodat differentiating between the two groups (high and low performers). Thesystem looks at this distribution by calculating means and variancescores to determine whether or not a statistically significantdifference exists. Seeing no significant difference between scoredistributions of the two groups, the system would de-select this scalefor inclusion in subsequent assessments. As well, the Ambition scale, inthis example, would not be used in the benchmark for the position.

A key difference between this process and other selection assessments isthe customization of the assessment instrument itself for each differentposition. Most other selection tools hay some form of benchmarkingprocess, but none tests only the scales relevant to the benchmark,opting, to test ALL scales and only compare relevant scale scores to abenchmark while ignoring the others. This is what makes this assessmentsystem dyna sic as opposed to static as all other assessments in thefield.

The Fit-score is calculated by totaling, and averaging the pointdifferentials for an individual's scale score and the relative distancethat score is from the benchmark range. The direction (above or below)of the distance from the score range in the benchmark is not acontributing factor to the score itself, hut does inform the systemregarding statements and descriptions to be offered in user reports. Forexample, looking specifically at FIG. 6:

Example 1 shows a benchmark of 3-5 with the individual score of 8. Thisindividual's fit score would calculate to approximately 70 (possible100) based upon the distance from the highlighted range and the stepwisealgorithm in place to weight distances based on the relative size of thebenchmark range.

Example 2 shows a benchmark of 7-10 with the individual score of 2. Thisindividual's fit score would calculate to approximately 30 since it isfurther from the benchmark and given the fact there are moreopportunities to be in the benchmark scale since it is wider (moreinclusive) by design.

In one non-limiting embodiment, there is a system that includes a bankof hundreds of psychometric behavioral and cognitive items intended tomeasure an individual's characteristics for the general purpose ofdetermining the best fit for employment in a specific position.Assessments administered for particular jobs may be customized basedupon extensive locally validated job analysis procedures and/or analysisof traits of existing employees who have been successful in theposition. The items used may be grouped into personality constructs, orscales, and/or cognitive reasoning scale(s). The specific scalesmeasured may be selected based on the results of the job analyses. Thisunique scale selection process improves the efficiency and relevance ofeach assessment and/or reduces administration time when compared toassessment systems which subject candidates to examination of all scalesregardless of the correlation of the scales to the job in question.

There may also be a cognitive reasoning section which also may or maynot be administered depending upon the findings of the job analysisprocedures. The items used to measure cognitive reasoning are generallybased upon extensive research into the types of cognitive tasksindividuals should be able to successfully complete at various levels ofability. While cognitive ability has been shown to be a good predictorof success in a job, there are certain positions for which ability levelmay be gleaned from other sources such as education level or otherextant test scores. The cognitive reasoning portion of the assessment isprovided as an option, but may not be administered to all jobcandidates, especially if those candidates have other proof of cognitiveability level.

Scales listed below are a non-limiting exemplary set of scales. Suchwere selected based upon a meta-analysis of the literature related toindividual behavioral traits most commonly investigated by employersduring the hiring process. Each scale is assessed as it relates to theparticular job for which an individual is applying. Therefore, dependingon the benchmark created to describe a successful employee in that job,desirable score may fall anywhere within the range from low to high.Simply put, high scores on all scales is not always the desired outcome.It is important to note that only those scales deemed relevant to thejob, based on the job benchmark, will be assessed.

The following is a non-limiting exemplary list of scales:Conscientiousness; Openness; Acumen; Assurance; Resolve; Reliability;Creativity; Extraversion; Intensity; Control; Decisiveness; Autonomy;Ambition; Adventurousness; Agreeableness; Gregariousness;Cooperativeness: Tact; Influence; Compassion; Teamwork; Stability;Compliance; Optimism; Happiness; Social Desirability; Problem Solving;and/or Cognitive Reasoning.

In one non-limiting embodiment, a system may include a scale and/or anautomated process for helping determine the viability of assessmentresults as a valid means of informing the applicant selection and/orplacement function. A response bias scale may measure the forthrightnessof the respondent and the statistical consistency of responses tin theassessment. When respondents intentionally try to misrepresentthemselves, have difficulty reading, or simply select item responsesrandomly the resulting inconsistency of responses can automatically leadto a warning in the report stating that the assessment results may notbe representative of the individual and should be used with caution. Theresponse bias scale does not determine whether or not a person is lying,but rather refers to the validity of responses and the confidence whichmay be placed on results based upon the individual's response patterns.

It is understood that the above-described embodiments are onlyillustrative of the application of the principles of the presentinvention. The present invention may be embodied in other specific formswithout departing from its spirit or essential characteristics. Thedescribed embodiment is to be considered in all respects only asillustrative and not restrictive. The scope of the invention is,therefore, indicated by the appended claims rather than by the foregoingdescription. All changes which come within the meaning and range ofequivalency of the claims are to be embraced within their scope.

For example, although the description herein often discusses modules andthe like with respect to an employer seeking candidates, the samemodules may be utilized by a prospective employee seeking a good fit ina job.

Additionally, although the figures illustrate a specific order toparticular steps, it is understood that the invention is not limited tothe illustrated order of steps herein.

It is also envisioned that a single system may include a plurality ofuser interfaces which may appear to service one particular group ofusers, but the entire system may support multiple such groups. As anon-limiting example, the same system may include functionality andmodules therewith to support prospective employees in search for goodfit jobs, while simultaneously facilitating employers in search for goodjob candidates and simultaneously assisting students in search ofmentors/tutors for particular subjects, and assisting funding sources infinding suitable projects to fund (e.g. research grants).

Thus, while the present invention has been fully described above withparticularity and detail in connection with what is presently deemed tobe the most practical and preferred embodiment of the invention, it willbe apparent to those of ordinary skill in the art that numerousmodifications, including, but not limited to, variations in size,materials, shape, form, function and manner of operation, assembly anduse may be made, without departing from the principles and concepts ofthe invention as set forth in the claims. Further, it is contemplatedthat an embodiment may be limited to consist of or to consistessentially of one or more of the features, functions, structures,methods described herein.

What is claimed is:
 1. An assessment system configured to automaticallyassess an individual and an associated job using a computing device,comprising: a) a benchmark module that stores a benchmark associatedwith a job; b) an electronic library of questions associated with anarray of characteristics; c) a survey module functionally coupled to theelectronic library and the benchmark module that automatically performsan assessment by automatically selecting questions from the electroniclibrary based on the benchmark and applying such questions to anindividual using a user interface module; and d) an analysis modulefunctionally coupled to the survey module that analyzes application ofthe assessment process to the individual.
 2. The system of claim 1,wherein the survey module automatically generates a benchmark associatedwith a job by applying an assessment to a pool of individuals thatperform that job.
 3. The system of claim 1, wherein the benchmarkincludes both minimum threshold scores and maximum threshold scoresassociated with particular characteristics.
 4. The system of claim 1,further comprising a benchmark library including electronic records ofan array of benchmarks against which the analysis module may compare anindividual.
 5. The system of claim 1, wherein the analysis moduleautomatically generates a repot that details the analysis of theindividual with respect to the job.
 6. The system of claim 1, furthercomprising an update module that modifies the benchmark based onreal-time employment data from an employer.
 7. A method of automaticallyassessing an individual and an associated job using a computing device,comprising the steps of: a) storing a benchmark associated with a job;b) associating a benchmark with a plurality of characteristics; c)providing an electronic library of questions wherein the questions areassociated with an array of characteristics including the plurality ofcharacteristics; d) automatically selecting questions from theelectronic library of questions based on the benchmark and itsassociated plurality of characteristics; and e) applying theautomatically selected questions to an individual.
 8. The method ofclaim 7, further comprising the step of automatically analyzingapplication of the assessment process to the individual using acomputing device.
 9. The method of claim 7, further comprisingautomatically generates a benchmark associated with a job by applying anassessment to a pool of individuals that perform that job.
 10. Themethod of claim 7, further comprising generating both minimum thresholdscores and maximum threshold scores associated with the plurality ofcharacteristics while associating the benchmark with the plurality ofcharacteristics.
 11. The method of claim 7, further comprising providinga benchmark library including electronic records of an array ofbenchmarks against which the analysis module may compare an individualand automatically applying automatically selected questions associatedwith each of the array of benchmarks to the individual.
 12. The methodof claim 7, further comprising automatically generates a report thandetails the analysis of the individual with respect to the job.
 3. Themethod of claim 7, further comprising modifying the benchmark based onreal-time employment data from an employer, including real-time dataregarding hiring and firing of employees within an employee poolassociated with a particular benchmark.
 14. The method of claim 8,further comprising automatically generates a benchmark associated with ajob by applying an assessment to a pool of individuals that perform thatjob.
 15. The method of claim 14, further comprising generating bothminimum threshold scores and maximum threshold scores associated withthe plurality of characteristics while associating the benchmark withthe plurality of characteristics.
 16. The method of claim 15, furthercomprising providing a benchmark library including electronic records ofan array of benchmarks against which the analysis module may compare anindividual and automatically applying automatically selected questionsassociated with each of the array of benchmarks to the individual. 17.The method of claim 16, further comprising automatically generates areport that details the analysis of the individual with respect to thejob.
 18. The method of claim 17, further comprising modifying thebenchmark based on real-time employment data from an employer, includingreal-time data regarding hiring and firing of employees within anemployee pool associated with a particular benchmark.
 19. An assessmentsystem configured to automatically assess an individual and anassociated job using a computing device over a computerized network,comprising: a) a benchmark module that stores a benchmark associatedwith a job, wherein the benchmark includes both minimum threshold scoresand maximum threshold scores associated with particular characteristics;b) an electronic library of questions associated with an array ofcharacteristics; c) a survey module functionally coupled to theelectronic library and the benchmark module that automatically generatesa benchmark associated with a job by applying an assessment to a pool ofindividuals that perform that job and that automatically performs anassessment by automatically selecting questions from the electroniclibrary based on the benchmark and applying such questions to anindividual using a user interface module over a computerized network; d)an analysis module functionally coupled to the survey module thatanalyzes application of the assessment process to the individual; 20.The system of claim 19, further comprising: e) a benchmark libraryincluding electronic records of an array of benchmarks against which theanalysis module may compare an individual; and f) an update module thatmodifies the benchmark based on real-time employment data from anemployer.